Filtering device and environment recognition system

ABSTRACT

A filtering device includes an evaluation value deriving module that derives, for a pair of images having mutual relevance, multiple evaluation values indicative of correlations between any one of blocks (reference block) that is extracted from one of the images (reference image) and multiple blocks (comparison blocks) extracted from the other image (comparison image), an evaluation range setting module that sets an evaluation range of the evaluation values, the evaluation range having one of boundaries at the evaluation value with the highest correlation among the multiple evaluation values, and a difference value determining module that determines whether the evaluation value with the highest correlation is valid as a difference value based on the multiple evaluation values and the evaluation range.

CROSS-REFERENCE TO RELATED APPLICATION

The present application claims priority from Japanese Patent ApplicationNo. 2013-205390 filed on Sep. 30, 2013, the entire contents of which arehereby incorporated by reference.

BACKGROUND

1. Technical Field

The present disclosure relates to a filtering device and an environmentrecognition system which determines, when calculating a difference value(parallax) of an object among multiple comparison targets, whether thedifference value is valid.

2. Related Art

There are conventionally known a technique, such as collision avoidancecontrol, which detects specific objects including another vehiclelocated ahead of a vehicle, and avoids a collision with a leadingvehicle, and a technique, such as a cruise control, which controls so asto maintain an inter-vehicle distance with a leading vehicle at a safedistance (for instance, see Japanese Patent (JP-B) No. 3,349,060).

Such a collision-avoidance control and cruise control derive a parallaxby using so-called pattern matching, in order to acquire a relativedistance from the vehicle, of an object located ahead the vehicle. Thepattern matching acquires image data from, for example, each of twoimaging devices of which viewpoints differ from each other. The patternmatching then extracts any one of blocks from an image (hereinafter,referred to as “the reference image”) based on the image data generatedby one of the imaging devices, and then searches a highly-correlatedblock from an image (hereinafter, referred to as “the comparison image”)based on the image data generated by the other imaging device. Then, thepattern matching refers to imaging parameters, such as installedpositions and focal lengths of the imaging devices, uses so-called astereo method or a triangulation method to calculate relative distancesof the object with respect to the imaging devices based on the derivedparallax, and converts the calculated relative distances intothree-dimensional (3D) positional information which contains ahorizontal distance and a height of the object in addition to thecalculated relative distances. Further, various recognition processingare performed using the 3D positional information. Note that the term“horizontal” as used herein refers to screen horizontal or lateraldirections, and the term “vertical (described later)” as used hereinrefers to screen vertical directions.

The above-described pattern matching calculates the correlation of theblock in the comparison image with the block in the reference imagewhile horizontally shifting the target block of the comparison image,and then calculates differences (difference values) between coordinatesof the block in the comparison image of which the correlation is highestand coordinates of the block in the reference image, as the parallax.However, in a case where similar patterns continue horizontally, thecorrelations hardly indicate differences between blocks even if thetarget block in the comparison image is changed or shifted, resulting inerroneous derivations of the parallaxes.

Therefore, for example, JP-B No. 3,348,939 discloses a technique thatuses an evaluation function in the pattern matching, such as sum ofabsolute difference (SAD) which is obtained by integrating thedifferences in luminance between each pixel within one block in thecomparison image and a corresponding pixel, located at the same positionas the first pixel, within another block in the reference image. Thistechnique determines whether the minimum value of the evaluation values(where the correlation becomes the highest value) satisfies apredetermined condition, for example, whether the minimum value is lessthan a predetermined fixed value. If the minimum value is less than thepredetermined fixed value, the parallax of the target block isdetermined to be valid. Further, for example, JP-B No. 3,917,285discloses a technique that calculates average values in luminance ofpixels around respective blocks in the reference image and thecomparison image, subtracts each average value from the luminance withinthe corresponding block to derive an evaluation value (i.e., averagevalue difference matching). By using such evaluation functions,variations in the images between the reference image and the comparisonimage, as well as effects due to low-frequency noise can be reduced.

However, the technique disclosed in JP-B No. 3,348,939 which comparesthe minimum value of the evaluation values with the fixed value has adifficulty to determine a unified or common fixed value which can beused for the evaluation of the existence of the correlation undervarious environments. Moreover, in the technique disclosed in JP-B No.3,917,285 using the average value difference matching, the evaluationvalue greatly varies depending on imaging conditions of the imagingdevices, for example, a fogging state of a windshield in front of theimaging devices and/or brightness of the circumference environment, evenif the comparing images are obtained under similar environments.Therefore, the parallax (difference value) to be invalidated has notbeen effectively excluded.

SUMMARY OF THE INVENTION

The present disclosure has been designed in consideration of thecircumstances described above, and an object thereof is to provide afiltering device and an environment recognition system, that caneffectively exclude a difference value to be invalidated byappropriately evaluating an evaluation value of an evaluation function.

According to one aspect of the present disclosure, a filtering device isprovided, which includes an evaluation value deriving module thatderives, for a pair of comparison targets having mutual relevance,multiple evaluation values indicative of correlations with any one ofextracted parts extracted from one of the comparison targets andmultiple extracted parts that are extracted from the other comparisontarget, respectively, an evaluation range setting module that sets anevaluation range of the evaluation values, the evaluation range havingone of boundaries at the evaluation value with the highest correlationamong the multiple evaluation values, and a difference value determiningmodule that determines whether the evaluation value with the highestcorrelation is valid as a difference value based on the multipleevaluation values and the evaluation range.

The difference value determining module may determine that theevaluation value with the highest correlation is invalid as thedifference value if a ratio of the evaluation values contained in theevaluation range to all the multiple evaluation values is greater than athreshold.

The difference value determining module may determine that theevaluation value with the highest correlation is invalid as thedifference value if an area of a polygon formed with the evaluationvalues contained in the evaluation range is greater than a threshold,the polygon being formed so as to have the other boundary of theevaluation range as one side of the polygon.

The evaluation range setting module may change the evaluation rangeaccording to the evaluation value with the highest correlation.

The comparison target may be an image and the extracted part may be ablock consisting of one or more pixels in the image.

The evaluation range setting module may change the evaluation rangeaccording to an average value of luminance of the pixels in apredetermined area of the image.

According to another aspect of the present disclosure, an environmentrecognition system is provided, which includes one or more imagingdevices that generate a pair of images having mutual relevance, anevaluation value deriving module that derives, for the pair of generatedimages, multiple evaluation values indicative of correlations betweenany one of blocks that is extracted from one of the images and multipleblocks extracted from the other image, an evaluation range settingmodule that sets an evaluation range of the evaluation values, theevaluation range having one of boundaries at the evaluation value withthe highest correlation among the multiple evaluation values, and adifference value determining module that determines whether theevaluation value with the highest correlation is valid as a parallaxbased on the multiple evaluation values and the evaluation range.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is illustrated by way of example and not by wayof limitation in the figures of the accompanying drawings, in which thelike reference numerals indicate like elements and in which:

FIG. 1 is a block diagram illustrating a connecting relation of anenvironment recognition system;

FIG. 2 is a functional block diagram schematically illustratingfunctions of a vehicle exterior environment recognition device;

FIGS. 3A to 3C illustrate average value difference matching;

FIG. 4 illustrates a reference image and a comparison image;

FIGS. 5A to 5C are graphs illustrating transitions of evaluation valuesin three arbitrary areas;

FIGS. 6A to 6C are graphs illustrating processing for determining theevaluation values; and

FIG. 7 is a graph illustrating another processing for determining theevaluation value.

DETAILED DESCRIPTION

Hereinafter, a suitable example of the present disclosure will bedescribed in detail with reference to the accompanying drawings.Dimensions, material, concrete numerical values, etc. illustrated inthis example are merely instances for easier understanding of thepresent disclosure, and, unless otherwise particularly specified, thoseinstances are not intended to limit the present disclosure. Note that,in this description and the accompanying drawings, elements having thesubstantially same function and configuration are denoted with the samereference numerals in order to omit redundant explanations, and anyother elements which are not directly related to the present disclosureare not illustrated in the accompanying drawings.

In recent years, vehicles having so-called a collision avoidancefunction (adaptive cruise control: ACC) have been spreading. Thisfunction images the road environment ahead of a vehicle where on-boardcameras are mounted, identifies an object, such as a leading vehicle,based on color information and/or positional information obtained fromthe images (comparison targets), and thereby avoiding a collision withthe identified object and/or maintains an inter-vehicle distance withthe leading vehicle at a safe distance.

In order to acquire the relative inter-vehicle distance of the objectlocated ahead of the vehicle, the collision avoidance function uses, forexample, two imaging devices of which viewpoints differ from each otherto acquire a pair of comparison targets (i.e., a reference image and acomparison image) from each of the two imaging devices, then usesso-called pattern matching to compare the comparison image with thereference image and extracts a highly-correlated block (extracted part).However, when similar patterns continue in the horizontal directions ofthe image, the correlations do not vary between adjacent blocks even ifthe position of the target block in the comparison image is changed,resulting in possible erroneous derivations of the parallaxes(difference values). Therefore, the purpose of this example is toeffectively exclude the parallaxes to be invalidated by determiningwhether the minimum value is valid as the parallax based on transitionsof evaluation values in an evaluation function of the pattern matching,i.e., transitions around the minimum value of the evaluation values(where the value of the correlation becomes the highest). Note that theterm “block” as used herein refers to a part comprised of one or morepixels in an image. Below, the environment recognition system forachieving such a purpose will be described, and a filtering deviceprovided to a vehicle exterior environment recognition device that is aparticular element of the system will be described in detail.

(Environment Recognition System 100)

FIG. 1 is a block diagram illustrating a connecting relation of anenvironment recognition system 100. The environment recognition system100 is comprised of a pair of imaging devices 110 mounted inside avehicle 1 (hereinafter, simply referred to as “the vehicle”), a vehicleexterior environment recognition device 120, and a vehicle controldevice 130 (which is typically comprised of an electronic control unit(ECU)).

Each imaging device 110 is comprised of image elements, such ascharge-coupled devices (CCDs) or complementary metal-oxidesemiconductors (CMOSs). Each imaging device 110 can image theenvironment ahead of the vehicle 1 to generate a color image consistingof three hues (R (red), G (green), and B (blue)) or a monochrome image.Note that the color image imaged by the imaging device 110 is adopted asthe luminance image to distinguish it from a distance image describedlater.

Moreover, the two imaging devices 110 are mounted so as to be separatedfrom each other in substantially lateral or horizontal directions suchthat they are oriented facing to the traveling direction of the vehicle1 to have their optical axes being oriented substantially parallel toeach other. Each imaging device 110 sequentially generates image datawhich is obtained by imaging objects existing within a detection areaahead of the vehicle 1 per frame, for example, at every 1/60 seconds(i.e., 60 fps). Note that the term “object” to be recognized as usedherein refers not only a solid object existing independently, such as avehicle, a pedestrian, a traffic light, a road surface (traveling path),a guardrail, and a building, but also an object that can be identifiedas part of the solid object, such as a taillight, a blinker, eachilluminating part of the traffic light. Each functional module describedbelow carries out processing for every frame, triggered by a refreshingof such image data.

The vehicle exterior environment recognition device 120 acquires imagedata from each of the two imaging devices 110, derives the parallaxusing so-called pattern matching, and associates the derived parallaxinformation (corresponding to a relative distance described later) withthe image data to generate a distance image. The pattern matching willbe described later in detail. Further, the vehicle exterior environmentrecognition device 120 uses the luminance based on the luminance imageand three-dimensional (3D) positional information in real spacecontaining the relative distances with respect to the vehicle 1 based onthe distance image to group blocks, of which the luminance are equal andthe 3D positional information are close to each other, as one unitaryobject, and then identifies specific objects to which the object in thedetection area ahead of the vehicle 1 corresponds.

When the specific object is identified, the vehicle exterior environmentrecognition device 120 derives a relative speed and the like of thespecific object (for example, leading vehicle) while tracking thespecific object, and then determines whether a possibility of thespecific object colliding with the vehicle 1 is high. Here, if thevehicle exterior environment recognition device 120 determines that thepossibility of a collision is high, the vehicle exterior environmentrecognition device 120 then gives (informs) a vehicle operator a warningindication through a display unit 122 installed in front of theoperator, and outputs information indicative of the warning to thevehicle control device 130.

The vehicle control device 130 accepts operational inputs of theoperator through a steering wheel 132, an accelerator (gas pedal) 134,and a brake pedal 136, and transmits the inputs to a steering mechanism142, a drive mechanism 144, and a brake mechanism 146, respectively, tocontrol the vehicle 1. The vehicle control device 130 also controls thedrive mechanism 144 and the brake mechanism 146 according toinstructions from the vehicle exterior environment recognition device120.

Next, a configuration of the vehicle exterior environment recognitiondevice 120 will be described in detail. Note that processing forobtaining the parallax of the object by the filtering device, which is afeature of this example, will be described in detail herein and, thusdescription of any other configurations unrelated to the feature of thisexample will be omitted.

(Vehicle Exterior Environment Recognition Device 120)

FIG. 2 is a functional block diagram schematically illustratingfunctions of the vehicle exterior environment recognition device 120. Asillustrated in FIG. 2, the vehicle exterior environment recognitiondevice 120 is comprised of an I/F unit 150, a data holding unit 152, anda central controlling unit 154.

The I/F unit 150 is an interface that performs bidirectional informationexchanges with the imaging devices 110 and the vehicle control device130. The data holding unit 152 is comprised of one ore more RAMs, oneore more flash memories, one ore more HDDs, etc. to hold variousinformation required for the processing of each functional moduledescribed below, and temporarily holds the image data received from theimaging devices 110.

The central controlling unit 154 is comprised of one ore more integratedcircuits containing one ore more central processing units (CPUs), oneore more ROMs where one or more programs and the like are stored, or oneor more RAMs as work areas, and controls the I/F unit 150, the dataholding unit 152, etc. through a system bus 156. In this example, thecentral controlling unit 154 also functions as an evaluation valuederiving module 160, an evaluation range setting module 162, and adifference value determining module 164 in this example. In addition,the evaluation value deriving module 160, the evaluation range settingmodule 162, and the difference value determining module 164 alsofunction as the filtering device. Next, the pattern matching processingwill be described in detail.

(Pattern Matching Processing)

The evaluation value deriving module 160 acquires the image data fromeach of the two imaging devices 110. The evaluation value derivingmodule 160 then derives a evaluation values based on the acquired twopieces of image data, which have mutual relevance. Specifically, theevaluation value deriving module 160 extracts any one of blocks from oneof the images (reference image) to be used as a reference, and extractsa multiple blocks from the other image (comparison image) to be used asa comparison target. The evaluation value deriving module 160 thenderives the multiple evaluation values indicative of correlationsbetween the reference block and the respective comparison blocks.

The pattern matching may be a comparison in luminance (Ycolor-difference signal) per block between the pair of images. Forexample, the pattern matching includes approaches, such as a sum ofabsolute difference (SAD) in which a difference in luminance iscalculated, a sum of squared intensity difference (SSD) which usesvalues obtained by squaring the differences, and a normalized crosscorrelation (NCC) which uses similarities of variances obtained bysubtracting an average value of the luminance of pixels from theluminance of each pixel. Among these, SAD will be particularly describedherein as an instance. In this example, average value differencematching is also performed. This average value difference matchingcalculates an average value of the luminance of pixels around a block inthe reference image and the comparison image, respectively, andsubtracts each average value from the luminance of the pixels within theblock to derive evaluation values. Next, the average value differencematching will be described in detail.

FIGS. 3A to 3C illustrate the average value difference matching. Asillustrated in FIG. 3A, the evaluation value deriving module 160 firstextracts a block 204 (hereinafter, referred to as “the reference block”)comprised of a matrix of pixels 202, for example, consisting of 4 pixelsin the horizontal directions×4 pixels in the vertical directions, fromthe reference image 200. The evaluation value deriving module 160sequentially repeats the processing per block, by extracting anotherreference block 204 and deriving the parallax for each extractedreference block 204. While the reference block 204 is comprised of 4pixels in the horizontal directions×4 pixels in the vertical directionsin this example; any number of pixels within the reference block 204 maybe selected.

One reference block 204 is extracted so as not to overlap with anotheradjacent reference block 204. In this example, the adjacent referenceblocks 204 are extracted, and thus all 6,750 blocks (150 blocks in thehorizontal directions×45 blocks in the vertical directions) aresequentially extracted as the reference block 204, for all the pixels202 displayed within the detection area (for example, 600 pixels in thehorizontal directions×180 pixels in the vertical directions).

Since the average value difference matching is adopted in this exampleas described above, the evaluation value deriving module 160 calculates,as illustrated in FIG. 3B, an average value Ab of luminance Rb(i, j) ofthe pixels 202 within an area 206 represented by 8 pixels in thehorizontal directions×8 pixels in the vertical directions around thereference block 204 centering on the reference block 204 based on thefollowing Equation 1:

Ab=ΣRb(i,j)/64  (Equation 1)

Note that is a horizontal pixel position in the area 206 (i=1 to 8 and“j” is a vertical pixel position in the area 206 (j=1 to 8). If the area206 is partially located outside the reference image 200 (i.e., the area206 is partially missing at an end of the reference image 200), theaverage value Ab is calculated while the missing part is omitted.

The evaluation value deriving module 160 then subtracts theabove-described average value Ab from the luminance Eb(i, j) of thepixels 202 within the reference block 204 to derive an average valuedifference luminance EEb(i, j) as the following Equation 2:

EEb(i,j)=Eb(i,j)−Ab  (Equation 2)

Note that “i” is a horizontal pixel position within the reference block204 (i=1 to 4), and “j” is a vertical pixel position within thereference block 204 (j=1 to 4).

The evaluation value deriving module 160 then extracts a block 214(hereinafter, referred to as “the comparison block”) represented by thematrix of pixels 212, for example, of 4 pixels in the horizontaldirections×4 pixels in the vertical directions from the comparison image210, as illustrated in FIG. 3C. Note that the evaluation value derivingmodule 160 sequentially extracts, for each one of the reference blocks204, multiple comparison blocks 214, and derives the evaluation valuesindicative of correlations with the respective reference blocks 204.

The comparison block 214 is shifted by, for example, 1 pixel at a timein the horizontal direction and then extracted and, thus, the pixels ofthe adjacent comparison blocks 214 are overlapped. In this example, thetotal of 128 comparison blocks 214 to the left and right in thehorizontal direction are extracted, for each one of the reference blocks204, with respect to a position 216 corresponding to the reference block204. Therefore, the extraction area (search area) has 131 pixels (=128+3pixels) in the horizontal directions×4 pixels in the verticaldirections. The positional relationship between the position 216corresponding to the reference block 204 and the extraction area is setaccording to the appearing pattern of the parallaxes between thereference image 200 and the comparison image 210.

Since the average value difference matching is adopted in this exampleas described above, the evaluation value deriving module 160 calculatesan average value Ac of luminance Rc(i, j) of pixels within an arearepresented by 8 pixels in the horizontal directions×8 pixels in thevertical directions around the comparison block 214 centering on thecomparison block 214 based on the following Equation 3, similar to thereference block 204:

Ac=ΣRc(i,j)/64  (Equation 3)

Note that “i” is a horizontal pixel position within the area (i=1 to 8),and “j” is a vertical pixel position within the area (j=1 to 8).

Next, the evaluation value deriving module 160 subtracts theabove-described average value Ac from luminance Ec(i, j) of each of thepixels within the comparison block 214 to derive an average valuedifference luminance EEc(i, j), as the following Equation 4:

EEc(i,j)=Ec(i,j)−Ac  (Equation 4)

Note that “i” is a horizontal pixel position within the comparison block214 (i=1 to 4), and “j” is a vertical pixel position within thecomparison block 214 (j=1 to 4).

Next, the evaluation value deriving module 160 subtracts, from theaverage value difference luminance EEb(i, j) of each pixel 202 of thereference block 204, the average value difference luminance EEc(i, j) ofeach pixel 212 corresponding to the same position in the comparisonblock 214, and integrates the subtraction results to derive anevaluation value S, as illustrated in the following Equation 5:

S=Σ(EEb(i,j)−EEc(i,j))  (Equation 5)

Thus, the multiple derived evaluation values S have higher correlationsas the evaluation values S themselves have smaller values, i.e., smallerdifferences. Therefore, among the multiple evaluation values (here, 128evaluation values) of one reference block 204 with the comparison block214, the position of the minimum evaluation value (minimum value),serves as a candidate of the position indicating an end of theparallaxes.

When the minimum value of the evaluation values for any reference block204 is derived, the evaluation value deriving module 160 holds, in apredetermined area of the data holding unit 152, a difference betweencoordinates of the comparison block 214 corresponding to the minimumvalue and coordinates of the reference block 204 (the position 216corresponding to the reference block 204) as the parallax. Therefore,the data holding unit 152 holds the parallaxes of 6,750 reference blocks204 (=150 blocks in the horizontal directions×45 blocks in the verticaldirections).

The above-described average value difference matching only useshigh-frequency components of the images for the matching, and can removelow-frequency noise because it has an equivalent function to a high-passfilter. In addition, the matching has high accuracy of identifying theparallaxes and thus can improve accuracy of deriving the parallaxes,even under the effects of slight imbalances in luminance between thereference image 200 and the comparison images 210, and the effects ofgain variations due to aging of the cameras (imaging devices) and/oranalog circuit components.

FIG. 4 illustrates the reference image 200 and the comparison image 210.The upper part (a) of FIG. 4 illustrates the image from one of theimaging devices 110 located on the right, while the lower part (b)illustrates the corresponding image from the other imaging device 110located on the left. In this example, the right image is used as thereference image 200, and the left image is used as the comparison image210.

When the evaluation value deriving module 160 derives the evaluationvalues based on the reference image 200 and the comparison image 210, itis easy to derive the parallaxes and perform the pattern matchingbecause an edge appears at a side end of a track in an area 220illustrated in the upper part (a) of FIG. 4.

On the other hand, if similar patterns continue horizontally like whitelines of a pedestrian crossing in an area 222 illustrated in the upperpart (a) of FIG. 4, the parallaxes may be erroneously derived since thecorrelation does not vary much between the blocks even if the positionsof the blocks in the comparison image 210 are different from the blocksof the reference image 200. Similarly, like the background, such as thesky, in an area 224 illustrated in the upper part (a) of FIG. 4, thecorrelation does not vary much between the blocks in the area wheresimilar features continue horizontally and, thus, the parallaxes may beerroneously derived as well. Other than the situations described above,a similar situation where the parallaxes cannot be identified may alsobe caused when occlusion occurs.

FIGS. 5A to 5C are graphs illustrating transitions of the evaluationvalues in the three areas 220, 222 and 224 described above. FIG. 5Aillustrates an instance of one reference block 204 in the area 220, FIG.5B illustrates an instance of one reference block 204 in the area 222,and FIG. 5C illustrates an instance of one reference block 204 in thearea 224. In FIGS. 5A to 5C, the horizontal axis represents thehorizontal position of the comparison block 214, and the vertical axisindicates the evaluation value. Smaller evaluation values indicatehigher correlations.

In FIG. 5A, since the blocks compared in the area 220 arecharacteristic, the evaluation value is outstanding locally toward asmaller value and, thus, a minimum value appears clearly. On the otherhand, in FIGS. 5B and 5C, minimum values of the evaluation valuecorresponding to the pattern matching appear as well; however, othersimilar evaluation values appear near the minimum values, respectively.

For this reason, in this example, the minimum value is determined to bevalid as the parallax, focusing on the transition of the evaluationvalue of the pattern matching, i.e., the shape of the transition aroundthe minimum value of the evaluation values (a value where thecorrelation becomes the highest). Specifically, if other evaluationvalues do not appear near the minimum value, the minimum value isdetermined to be valid as the parallax, and, on the other hand, otherevaluation values appear near the minimum value, the minimum value isdetermined to be invalid. Next, this processing will be described indetail.

FIGS. 6A to 6C are graphs illustrating the processing for determiningthe evaluation value. Note that FIGS. 6A to 6C correspond to FIGS. 5A to5C, respectively. The evaluation range setting module 162 sets anevaluation range of the evaluation value from the minimum value amongthe multiple evaluation values (here, 128 evaluation values) (i.e., theevaluation value with the highest correlation) as one of boundaries(lower boundary) of the evaluation range so that the evaluation rangehas a certain width toward higher values (toward lower correlations)from the lower boundary. Here, the width or the size of the evaluationrange is conveniently fixed as illustrated in FIGS. 6A to 6C. In FIGS.6A to 6C, since one of the boundaries of the evaluation rangecorresponds to the minimum value, one can understand that the positions(heights in the graphs) of the evaluation range differ according to thetransitions of the evaluation values.

The difference value determining module 164 determines whether theevaluation value with the highest correlation is valid as the parallaxbased on the multiple evaluation values and the evaluation range. Forexample, the difference value determining module 164 determines, for allthe multiple evaluation values, whether the evaluation value with thehighest correlation is valid as the parallax based on a ratio of theevaluation values contained within the evaluation range.

Specifically, suppose that the minimum value of the evaluation values isMin and the width of the evaluation range is W, the number of times Nwhen the evaluation value K satisfies the following Equation 6 iscounted:

K<(Min+W)  (Equation 6)

If the counted number N is greater than a threshold, the differencevalue determining module 164 concludes that the minimum value has lowreliability to be the parallax and that the minimum value is invalid asthe parallax. Note that the threshold is determined by multiplying thenumber of comparison blocks 214 (i.e., 128 blocks) by a predeterminedratio (for example, 20%).

Alternatively, the difference value determining module 164 may determinewhether the evaluation value with the highest correlation is valid asthe parallax based on a polygonal area formed by the evaluation valuescontained in the evaluation range where one side of the polygon isformed by the other boundary of the evaluation range, instead of usingthe ratio of the evaluation value contained in the evaluation range.

FIG. 7 is a graph illustrating other processing for determining theevaluation values. FIG. 7 corresponds to FIG. 6B. The evaluation rangesetting module 162 connects adjacent evaluation values contained in theevaluation range among the multiple evaluation values (here, 128evaluation values), also connects a boundary different from the minimumvalue of the evaluation values in the evaluation range, to form thepolygon illustrated by hatching in FIG. 7. If the polygon area isgreater than the threshold, the difference value determining module 164concludes that the evaluation value with the highest correlation isinvalid as the parallax. The transition of the evaluation value near theminimum value can also be determined by using such an area.

In the above, the width of the evaluation range is temporarily fixed;however, the evaluation range setting module 162 may also adaptivelychange the evaluation range. For example, the evaluation range settingmodule 162 changes the evaluation range according to an average value ofluminance of pixels within a predetermined area of the reference image200 or the comparison image 210. Note that the predetermined area may beset variously, such as the entire image, an area having a predeterminedsize at a predetermined position, and an extracted area from thecomparison image 210.

Specifically, if the average value of the luminance of the pixels withinthe predetermined area is low, the evaluation range setting module 162sets the evaluation range smaller, and, on the other hand, if theaverage value is high, the evaluation range setting module 162 sets theevaluation range larger. This is because, in a situation where theluminance is entirely low, for example, during twilight time and atnight, the luminance values on the image become entirely low, and theevaluation value does not change much near the minimum value. Therefore,if the evaluation range is fixed, the parallaxes which areunintentionally excluded may increase.

Alternatively, the evaluation range setting module 162 may change theevaluation range according to the evaluation value with the highestcorrelation. For example, suppose that the minimum value of theevaluation values is Min, the maximum value of the evaluation value isMax, the scale of the evaluation value is Scale, and the counting numberis Gain (any value), the width W of the evaluation range is set based onfollowing Equation 7 or 8:

W=Min×Gain/Scale  (Equation 7)

W=(Max−Min)×Gain/Scale  (Equation 8)

By determining the width W of the evaluation range by using Equation 7or 8, the evaluation range can be set according to the minimum value andthe scale of the evaluation value. Therefore, the shape of thetransition of the evaluation value can be appropriately determined.

As described above, the evaluation range that is relatively changed notbased on a fixed value but based on the minimum value is used as thedetermination criterion in this example. Therefore, even if theluminance is varied depending on the fogging state of the windshield infront of the imaging devices 110 and/or the brightness of thecircumference environment, the parallaxes to be invalidated can beeffectively excluded.

In addition, the tendency of the evaluation value can be determinedbased on the entire shape of the transition of the evaluation value byusing the multiple points near the minimum value (the transitionitself), without using the minimum value of the evaluation values onlyas the evaluation criterion. Thus, the parallaxes to be invalidated,which can be determined due to the fact that they have no localdepression of the evaluation value, can be reliably excluded.

In addition, one ore more programs that cause one or more computers tofunction as the filtering device and/or the environment recognitionsystem described above, and one or more storage media, such as one ormore flexible discs, one or more magneto-optic discs, one or more ROMs,one or more CDs, one or more DVDs, and one or more BDs, which record theprogram(s) and can be read by one ore more computer, are also provided.Note that the term “program” as used herein refers to a data set that isdescribed with any of known languages and any of known describingmethods.

As described above, although the suitable example of the presentdisclosure is described with reference to the accompanying drawings, thepresent disclosure is not intended to be limited to this example. It isapparent that a person skilled in the art can reach various kinds ofchanges and/or modifications without departing from the scope of theappended claims, and it should be understood that those changes and/ormodifications naturally encompass the technical scope of the presentdisclosure.

For example, in the example described above, the pair of imagessimultaneously imaged by the two imaging devices 110 having differentviewpoints, as comparison targets. However, the present disclosure iswidely applicable to a pair of images having mutual relevance, withoutlimiting to the case of this example. Such a pair of images havingmutual relevance includes, for example, two images sequentiallyoutputted from one imaging device (e.g., a monocular camera) whichcaptured the images at different timings (which are processing targetsof so-called an optical flow), and a combination of the captured imagedand an image prepared in advance (which is processing targets ofso-called template matching). Further, the “parallax” between the pairof images which are simultaneously imaged by the two imaging devices 110having different viewpoints is used as the difference value in theexample described above. However, the difference value may be anydifference between corresponding extracted parts, such as a differencebetween the reference blocks in the pair of images having mutualrelevance, without limiting to the example described above.

Further, the luminance images are used as the comparison targets and theevaluation values are derived based on the luminance of the luminanceimage in the example described above. However, the evaluation values maybe derived based on information other than the luminance, for example,heat distribution acquired by a far-infrared camera, distribution ofreflection intensities acquired by laser radar, millimeter wave radar,etc., by using the information as the comparison targets, withoutlimiting to the luminance. Also in such cases, the difference value maysimilarly be the difference between the corresponding extracted parts.

The present disclosure is applicable to the filtering device and theenvironment recognition system which determine, when the differencevalues (parallaxes) of the object in the multiple images are calculated,whether the difference values are valid.

1. A filtering device, comprising: an evaluation value deriving modulethat derives, for a pair of comparison targets having mutual relevance,multiple evaluation values indicative of correlations with any one ofextracted parts extracted from one of the comparison targets andmultiple extracted parts that are extracted from the other comparisontarget, respectively; an evaluation range setting module that sets anevaluation range of the evaluation values, the evaluation range havingone of boundaries at the evaluation value with the highest correlationamong the multiple evaluation values; and a difference value determiningmodule that determines whether the evaluation value with the highestcorrelation is valid as a difference value based on the multipleevaluation values and the evaluation range.
 2. The filtering device ofclaim 1, wherein the difference value determining module determines thatthe evaluation value with the highest correlation is invalid as thedifference value if a ratio of the evaluation values contained in theevaluation range to all the multiple evaluation values is greater than athreshold.
 3. The filtering device of claim 1, wherein the differencevalue determining module determines that the evaluation value with thehighest correlation is invalid as the difference value if an area of apolygon formed with the evaluation values contained in the evaluationrange is greater than a threshold, the polygon being formed so as tohave the other boundary of the evaluation range as one side of thepolygon.
 4. The filtering device of claim 1, wherein the evaluationrange setting module changes the evaluation range according to theevaluation value with the highest correlation.
 5. The filtering deviceof claim 2, wherein the evaluation range setting module changes theevaluation range according to the evaluation value with the highestcorrelation.
 6. The filtering device of claim 3, wherein the evaluationrange setting module changes the evaluation range according to theevaluation value with the highest correlation.
 7. The filtering deviceof claim 1, wherein the comparison target is an image and the extractedpart is a block consisting of one or more pixels in the image.
 8. Thefiltering device of claim 2, wherein the comparison target is an imageand the extracted part is a block consisting of one or more pixels inthe image.
 9. The filtering device of claim 3, wherein the comparisontarget is an image and the extracted part is a block consisting of oneor more pixels in the image.
 10. The filtering device of claim 7,wherein the evaluation range setting module changes the evaluation rangeaccording to an average value of luminance of the pixels in apredetermined area of the image.
 11. The filtering device of claim 8,wherein the evaluation range setting module changes the evaluation rangeaccording to an average value of luminance of the pixels in apredetermined area of the image.
 12. The filtering device of claim 9,wherein the evaluation range setting module changes the evaluation rangeaccording to an average value of luminance of the pixels in apredetermined area of the image.
 13. An environment recognition system,comprising: one or more imaging devices that generate a pair of imageshaving mutual relevance; an evaluation value deriving module thatderives, for the pair of generated images, multiple evaluation valuesindicative of correlations between any one of blocks that is extractedfrom one of the images and multiple blocks extracted from the otherimage; an evaluation range setting module that sets an evaluation rangeof the evaluation values, the evaluation range having one of boundariesat the evaluation value with the highest correlation among the multipleevaluation values; and a difference value determining module thatdetermines whether the evaluation value with the highest correlation isvalid as a parallax based on the multiple evaluation values and theevaluation range.